Method of reconstructing three dimensional image using structured light pattern system

ABSTRACT

A method of reconstructing a three dimensional image using a structured light pattern system is provided as follows. A class identifier of an observed pixel on a captured image by a camera is extracted. The observed pixel has a coordinate (x, y) on the captured image. A first relative position of the x coordinate of the observed pixel in a tile domain of the captured image is calculated. A second relative position of one of a plurality of dots in a tile domain of a reference image using the extracted class identifier is calculated. A disparity of the observed pixel using the first relative position and the second relative position is calculated.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority of U.S. Provisional Application No.62/701,500 filed on Jul. 20, 2018, the disclosure of which isincorporated herein by reference in its entirety.

TECHNICAL FIELD

The present inventive concept relates to a method of reconstructing athree dimensional image using a structured light pattern system.

DISCUSSION OF RELATED ART

Structured light systems are used to reconstruct a three dimensionalimage of an object. To calculate a disparity map, the structured lightsystems perform various stereo matching algorithms that correlatepatches of pixels in the captured image of the camera and the referenceimage of the projector, which demand massive computation for comparingand selecting a best-matched patch with excessive power consumption. Thestereo matching algorithms may continue to access a memory storing thereference image while they are in operation.

SUMMARY

According to an exemplary embodiment of the present inventive concept, amethod of reconstructing a three dimensional image using a structuredlight pattern system is provided as follows. A class identifier of anobserved pixel on a captured image by a camera is extracted. Theobserved pixel has a coordinate (x, y) on the captured image. A firstrelative position of the x coordinate of the observed pixel in a tiledomain of the captured image is calculated. A second relative positionof one of a plurality of dots in a tile domain of a reference imageusing the extracted class identifier is calculated. A disparity of theobserved pixel using the first relative position and the second relativeposition is calculated.

According to an exemplary embodiment of the present inventive concept, amethod of reconstructing a three dimensional image using a structuredlight pattern system is provided as follows. A projected image to beprojected onto an object is generated from a reference image. Thereference image includes a plurality of tiles. Each of the tilesincludes a plurality of dots each of which is assigned to one of aplurality of class identifiers. A captured image is generated from theprojected image onto the object. A class identifier of an observed pixelon the captured image is generated. A first relative position of theobserved pixel in a tile domain of the captured image is calculated. Asecond relative position of one of the plurality of dots in a tiledomain of the reference image is calculated using the extracted classidentifier and a y coordinate of the observed pixel. A disparity of theobserved pixel is calculated using the first relative position and thesecond relative position.

A method of reconstructing a three dimensional image using a structuredlight pattern system is provided as follows. A reference image includinga plurality of tiles is generated. Each of the tiles includes aplurality of dots each of which is assigned to one of a plurality ofclass identifiers. A projected image generated from the reference imageis projected onto an object having a three-dimensional surface. Theprojected image onto the object is captured to generate a capturedimage. A class identifier from an observed pixel on the captured imageis extracted. The observed pixel is translated into a tile domain of thecaptured image to generate a first relative position in the tile domainof the captured image. A second relative position of one of theplurality of dots in a tile domain of the reference image is calculatedusing the extracted class identifier. A disparity of the observed pixelis calculated using the first relative position and the second relativeposition. A three-dimensional profile of the object is generated usingthe disparity.

BRIEF DESCRIPTION OF DRAWINGS

These and other features of the present inventive concept will becomemore apparent by describing in detail exemplary embodiments thereof withreference to the accompanying drawings of which:

FIG. 1 shows a block diagram of a structured light pattern systemaccording to an exemplary embodiment of the present inventive concept;

FIG. 2 shows a process flow for generating a depth map using astructured light pattern system according to an exemplary embodiment ofthe present inventive concept;

FIG. 3 shows a reference image having structured-light patternsaccording to an exemplary embodiment of the present inventive concept;

FIG. 4A shows a plurality of class identifiers assigned tostructured-light patterns of a reference image according to an exemplaryembodiment of the present inventive concept;

FIG. 4B shows a plurality of class identifiers assigned in a sub-dotscale to structured-light patterns of a reference image according to anexemplary embodiment of the present inventive concept;

FIG. 5 shows a distribution of a plurality of class identifiers on atile domain of a reference image according to an exemplary embodiment ofthe present inventive concept;

FIG. 6 shows rotation of structured patterns in a reference image at apredetermined rotation angle θ according to an exemplary embodiment ofthe present inventive concept; and

FIG. 7 shows a captured image of an object according to an exemplaryembodiment of the present inventive concept.

It will be appreciated that for simplicity and clarity of illustration,elements illustrated in the drawings have not necessarily been drawn toscale. For example, the dimensions of some of the elements areexaggerated relative to other elements for clarity. Further, whereconsidered appropriate, reference numerals have been repeated among thedrawings to indicate corresponding or analogous elements.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Exemplary embodiments of the present inventive concept will be describedbelow in detail with reference to the accompanying drawings. However,the inventive concept may be embodied in different forms and should notbe construed as limited to the embodiments set forth herein.

FIG. 1 shows a block diagram of a structured light pattern system 100.The structured light pattern system 100 includes a projector 101, acamera 102 and a processor 103. The structured light pattern system 100may be a stand-alone system. The present inventive concept is notlimited thereto. For example, the structured light pattern system 100may be part of a handheld device, such as, but not limited to, asmartphone, a cellphone or a digital camera.

The processor 103 sends a reference image 104 having a structured-lightpattern to the projector 101 projecting the reference image 104 onto anobject 105. The object 105 is represented by a line for simplicity ofdescription. The object 105 may include, but not limited to, a humanface or a scene, for example.

The processor 103 may be a microprocessor or a programmed softwarecodes, a dedicated integrated circuit or a combination of both. Forexample, the processor 103 may operate according to codes implementedcompletely via software, via software accelerated by a graphicsprocessing unit (GPU) or a multicore system. The processor 103 mayinclude a dedicated hardware implemented for the processing operations.Both hardware and software configurations may provide different stagesof parallelism.

The projector 101 and the camera 102 may be matched in the visibleregion or in the infrared light spectrum, which may not be visible tohuman eyes. For the convenience of description, it is assumedhereinafter that the projector 101 and the camera 102 are matched in theinfrared light spectrum.

The projector 101 may generate an infrared light having a projectedimage from the reference image 104 and illuminate the projected imageonto the object 105. The projector 101 may include a light source, suchas a laser diode or light-emitting-diode (LED). The projected image mayhave a structured-light pattern rotated from that of the reference image104 at a predetermined rotation angle. The rotation will be described inmore detail with reference to FIG. 6.

The camera 102 captures the projected image onto the object 105 andgenerates a captured image 106. The camera 102 may be an infrared camerahaving an infrared image sensor. The captured image 106 is transmittedto the processor 103, and the processor 103 generates a depth map 107 ofthe object 105.

According to an exemplary embodiment, the processor 103 may calculate adisparity without correlating the reference image 104 and the capturedimage 106, thereby reducing the amount of computation and powerconsumption, and thereby avoiding access to a memory storing thereference image 104.

Hereinafter, it will be described with reference to FIGS. 2 to 7 thathow the structured light pattern system 100 operates to generate thedepth map 107 of the object 105.

FIG. 2 shows a process flow for generating the depth map 107 using thestructured light pattern system 100. FIG. 3 shows the reference image104 having structured-light patterns. FIG. 4A shows a plurality of classidentifiers assigned to structured-light patterns of the reference image104. FIG. 4B shows a plurality of class identifiers assigned in asub-dot scale to structured-light patterns of the reference image 104.FIG. 5 shows a distribution of a plurality of class identifier on a tiledomain of the reference image 104. FIG. 6 shows rotation ofstructured-light patterns in the reference image 104 at a predeterminedrotation angle θ. FIG. 7 shows the captured image 106 of the object 105.The “tile domain” of the reference image 104 represents relativecoordinates on which the plurality of dots of each tile are distributedwith a unique relative location.

In the structured light pattern system 100, no matching operations oraccess to the reference image 104 is performed at runtime. In contrast,a structured light pattern system that is used in the industry mayperform a matching operation to identify, using various stereo matchingalgorithms, a conjugate pixel of the reference image 104 correspondingto an observed pixel of the captured image 106. The matching algorithmsmay demand massive computation for comparison. The matching algorithmsmay also cause a high input/output (I/O) access to a memory storing thereference image 104. For example, the Kinetic performs the PrimeSensealgorithm that extracts observed dots, then matches a patch around eachdot, with corresponding patches in the reference image using thenormalized cross correlation (NCC) algorithm. The NCC algorithm performsabout twenty (20) NCC patch matches per pixel, with the minimum NCCscore being selected. Then a sequential region growing process isperformed to create a dense disparity map. Each of dot extraction, NCCpatch matching, disparity selection and region growing takesconsiderable time with many pixel lookup and operations.

According to an exemplary embodiment, a disparity (d) is calculated asfollows:d=x′−x_ref  [Equation 1]

In Equation 1, x′ represents a first relative position of an observedpixel p_(c) (x, y) within a tile domain of the captured image 106, anobserved position x of the observed pixel p_(c) (x, y) being translatedinto the first relative position x′ (hereinafter, x′ will also bereferred to as a translated position x′), x is a value of x coordinatein a row (or a horizontal direction) of the captured image 106 and y isa value of y coordinate in a column (or a vertical direction) of thecaptured image 106.

In Equation 1, x_ref represents a second relative position of a dotcorresponding to the observed pixel p_(c) (x, y) within a tile domain ofthe reference image 104, the second relative position x_ref of the dotbeing calculated using a function derived from a distribution of aplurality of class identifiers assigned to a plurality of dots of thereference image 104 (hereinafter, x_ref will be referred to as aconjugate reference position x_ref).

The distribution of the class identifiers are fitted into Equation 2,and the function may be expressed as Equation 3 or Equation 4. Thefunctions expressed in Equations 2, 3 and 4 each may represent adistribution of the plurality of class identifiers on the tile domain ofthe reference image 104. The Equations 2, 3 and 4 will be describedbelow.

With reference to FIGS. 2 to 7, the inventive concept described abovewill be further described. Starting with step S210 in FIG. 2, theprocessor 103 generates the reference image 104 to be illuminated ontothe object 105. The reference image 104, as shown in FIG. 3, includes atile 300 that is repeated or tiled in both horizontal direction andvertical direction to completely fill the reference image 104. The tile300 is repeated ten times in the horizontal direction and 160 times inthe vertical direction for the projector 101 having a VGA resolution,for example.

The tile 300 may include a plurality of dots. For example, the tile 300includes 48 dots arranged in the horizontal direction, and four dotsarranged in the vertical direction. In this case, the tile 300 includes196 (48×4) dots. Each dot (black or white) has surrounding black orwhite dots uniquely arranged such that each dot and its surroundingblack or white dots may constitute a unique pattern in the tile 300. Fora pattern having 4×4 dots, there are a total of 192 unique patterns inthe tile 300.

FIG. 4A shows a plurality of class identifiers id_z assigned tostructured light patterns of the reference image 104. In an exemplaryembodiment, the structured light patterns may include the plurality ofdots uniquely arranged as shown in tile 300 of FIG. 3. However, thepresent inventive concept is not limited thereto. For example, thestructured light patterns may include various patterns. For example, thestructured light patterns may include a binary encoded pattern or astar-like pattern.

In FIG. 4A, each of the class identifiers id_z is assigned to one of theplurality of dots in the tile 300 of the reference image 104. Since thetile 300 has 192 unique patterns, each pattern (or each dot) may beclassified with a unique class identifier among the class identifiersid_z having values from 1 to 192. The class identifiers id_z areassigned to the dots left-to-right in a row, then continue to beassigned left-to-right in the next row until the class identifiers id_zall are assigned to the dots in the tile 300. The class identifiers id_zmay be integers, and the integer values of the class identifiers id_zsuccessively increase.

FIG. 4B shows a plurality of class identifiers assigned in a sub-dotscale to structured-light patterns of the reference image 104. To avoidquantization of disparity maps, each of the class identifiers id_z needto support sub-dot shifts of the structured-light patterns on thecaptured image 106. For sub-dot accuracy, each dot is additionallydivided into a plurality of sub-dots, equal to the desired level ofsub-dot precision. For example, each dot may be divided into 3×3sub-dots such that the tile 300 includes 1728 (4×48×16) unique sub-dots.A plurality of class identifiers having a value from 1 to 1728 areassigned to each of the unique sub-dots in the same manner as describedwith reference to FIG. 4A. For the convenience of description, FIG. 4Bshows partially the plurality of class identifiers.

In FIGS. 4A and 4B, each dot having its corresponding class identifierid_z is represented by a point uniquely defined in the tile domain ofthe reference image 104. In other words, the class identifiers id_z aredistributed over the tile domain of the reference image 104. Forexample, the distribution of the class identifiers id_z in FIG. 4B maybe visualized as shown in FIG. 5.

In FIG. 5, the class identifiers id_z are represented by values in thez-axis. The distribution of the class identifiers id_z may be fitted toa fitting function ID (y, x_ref) such that the fitting function ID (y,x_ref) may represent the distribution of the class identifiers over thetile domain of the tile 300.

For example, the distribution of the class identifiers id_z over thedomain of the tile 300 may be fitted into the fitting function ID (y,x_ref) expressed as follows:z=ID(y,x_ref)  [Equation 2]where z is a class identifier having an integer value between 1 to 1728,x_ref represents the second relative position in a row within the tiledomain of the reference image 104, and y represents a position in thereference image 104.

Equation 2 may be reformulated to calculate the second relative positionx_ref as follows:x_ref=F _(x)(y,z)F _(x)(y,z)=a*(z−mod(z−1,4*s)−1)+b*y  [Equation 3]

-   -   where,    -   y is a y coordinate in the reference image 104 that may be        referred as a row number, which corresponds to a y coordinate of        the captured image 106,    -   s is a horizontal dot sampling rate which corresponds to a        number of pixels arranged horizontally in a dot,    -   a=1/(4s·sin θ),    -   b=1/tan θ,    -   θ is a rotation angle which will defined with reference to FIG.        6, and    -   z is a pixel class identifier that is extracted in step S240.

In Equation 3, the “mod” function returns the remainder after divisionof (z−1) by (4*s), where (z−1) is the dividend and (4*s) is the divisor.For y, z, s and θ, the reformulated function (y, z) of Equation 3returns the second relative position x_ref within the tile domain of thereference image 104.

Alternatively, Equation 2 may be reformulated to calculate the secondrelative position x_ref as follows:

$\begin{matrix}{\mspace{20mu}{{{x\_ ref} = {F_{x}\left( {y,z} \right)}}{{F_{x}\left( {y,z} \right)} = {{a*\left( {z - {\cos\;\frac{\pi}{2s}z} + {\sin\;\frac{\pi}{2s}z} - {\frac{1}{2s}{\cos\left( {s\;\pi\; z} \right)}} - 1} \right)} + {b*y}}}}\;} & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack\end{matrix}$

-   -   where,    -   y is a y coordinate in the reference image 104    -   s is a horizontal dot sampling rate,    -   z is a pixel class identifier that is extracted in step S240,    -   a=1/(4s·sin θ), and    -   b=1/tan θ.

For y, z, s and θ, the reformulated function F_(x) (y, z) of Equation 4returns the second relative position x_ref the tile domain of thereference image 104.

In step S220, the projector 101 generates a projected image 400 from thereference image 104 and illuminates the projected image 400 onto theobject 105. The plurality of dots of the reference image 104 aretransferred to the projected image 400. In an exemplary embodiment, theprojected image 400 may be rotated at a predetermined rotation angle θas shown in FIG. 6. In an exemplary embodiment, the predeterminedrotation angle θ may be 12 degree, for example. The present inventiveconcept is not limited thereto. For example, the predetermined rotationangle θ may be an angle below 90 degree. [Please elaborate on whateffects are obtained by the rotation.]

In step S230, the camera 102 captures the projected image 400 onto theobject 105 and generates the captured image 106. In FIG. 7, an exemplarycaptured image is shown. The object 105 that is three-dimensional maydeform the projected image 400 along a three-dimensional surface profileof the object 105.

The resolutions of the projector 101 and the camera 102 may bedifferent. For example, the projector 101 may project the referenceimage 104 in a video graphics array (VGA) resolution (e.g., 640×480pixels), and the camera 102 may have a resolution that is higher (e.g.,1280×720 pixels). The captured image 106 may be down-sampled and/or onlythe area illuminated by the projector 101 may be analyzed to generatethe depth map 107.

In FIG. 7, the observed pixel p_(c) (x, y) may need translation into thetile domain of the captured image 106, thereby generating the firstrelative position x′ within the tile domain. Since the second relativeposition x_ref is a relative position within the tile domain of thereference image 104, rather than the absolute position in an entire rowin the reference image 104, the observed position x of the observedpixel p_(c) (x, y) need to be translated to the tile domain of thecaptured image 106. The “tile domain” of the captured image 106represents relative coordinates on which the plurality of projected dotsof each projected tile onto the object 105 are distributed with a uniquerelative location within each projected tile of the captured image 106.The translation will be described in detail with reference to step S250.

In step S240, the processor 103 analyzes the captured image 106 toextract a class identifier id_z of the observed pixel p_(c) (x, y), forexample, from the captured image 106. The processor 103 may scan thecaptured image 106 as shown in FIG. 7 line by line. The processor 103moves a window 500 along a scan line 600, thereby extracting a classidentifier within a pixel of the window 500 using various algorithms.Examples of those various algorithms to extract a class identifier areprovided in U.S. patent application Ser. No. 15/907,242 to Shi et al.,of which specification is entirely incorporated herein by reference.

In step S250, the processor 103 translates the observed pixel p_(c) (x,y) into the tile domain of the captured image 106, thereby generatingthe first relative position x′ for the observed pixel p_(c) (x, y). Inan exemplary embodiment, the following equation may be used to calculatethe first relative position x′ of the observed pixel p_(c) (x, y):x←F((x+x_offset) % (48*s))*cos θ−((y+y_offset) % (48*s*b))*sinθ,  [Equation 5]

-   -   wherein:    -   x is a x coordinate in the captured image 106,    -   y is a y coordinate in the captured image 106    -   % is a remainder operator,    -   s is a horizontal dot sampling rate,    -   z is a class identifier for the observed pixel p_(c) (x, y),    -   b=1/tan θ, and    -   x_offset and y_offset that are system parameter set by a        calibration of the structured light pattern system 100.

In step S260, the processor 103 calculates the second reference positionx_ref using the Equation 3 or Equation 4. With a row number (or ycoordinate) of the observed pixel p_(c) (x, y) in the captured image 106and the class identifier id_z obtained in step S240, the Equation 3returns the second relative position x_ref of a dot on the referenceimage 104 corresponding to the observed pixel p_(c) (x, y) on thecaptured image 106.

In step S270, the processor 103 calculates a depth map using a disparityto reconstruct the three-dimensional surface profile of the object 105.With the first relative position x′ on the captured image 106 and thesecond relative position x_ref calculated from Equation 3 or 4, theprocessor 103 may calculate a disparity d according to the Equation 1,and reconstruct a three-dimensional surface profile of the object 105using the disparity (d).

While the present inventive concept has been shown and described withreference to exemplary embodiments thereof, it will be apparent to thoseof ordinary skill in the art that various changes in form and detail maybe made therein without departing from the spirit and scope of theinventive concept as defined by the following claims.

What is claimed is:
 1. A method of reconstructing a three dimensionalimage using a structured light pattern system, the method comprising:extracting a class identifier of an observed pixel on a captured imageby a camera, wherein the observed pixel has a coordinate (x, y) on thecaptured image; calculating a first relative position of the xcoordinate of the observed pixel in a tile domain of the captured image;calculating a second relative position of one of a plurality of dots ina tile domain of a reference image using the extracted class identifier;and calculating a disparity of the observed pixel using the firstrelative position and the second relative position.
 2. The method ofclaim 1, wherein the calculating of the first relative positionincludes: translating the x coordinate of the observed pixel into thetile domain of the captured image to generate the first relativeposition in the tile domain of the captured image.
 3. The method ofclaim 1, further comprising: generating a projected image to beprojected onto an object from the reference image, wherein the referenceimage includes a plurality of tiles, wherein each of the tiles includesthe plurality of dots each of which is assigned to one of a plurality ofclass identifiers and wherein the extracted class identifier is one ofthe plurality of class identifiers; and generating the captured imagefrom the projected image onto the object.
 4. The method of claim 3,wherein the projected image is projected as infrared light.
 5. Themethod of claim 3, wherein the plurality of class identifiers each isassigned to one of the plurality of dots left to right and row-by-rowwithin the tile domain of the reference image, wherein the plurality ofclass identifiers have continuously-increasing values from a firstcorner of the tile domain of the reference image to a second corner ofthe tile domain of the reference image facing diagonally the firstcorner.
 6. The method of claim 3, wherein the calculating of the secondrelative position is performed by using a function of which an inputincludes the extracted class identifier and an output is the secondrelative position, and wherein the function represents a distribution ofthe plurality of class identifiers on the tile domain of the referenceimage.
 7. The method of claim 6, wherein the input of the functionfurther includes the y coordinate of the observed pixel.
 8. The methodof claim 7, wherein the plurality of dots of the reference image isrotated at a predefined rotation angle in the projected image, andwherein the input of the function further includes the predefinedrotation angle.
 9. The method of claim 8, wherein the input of thefunction further includes a dot sampling rate, and wherein the dotsampling rate corresponds to a number of pixels horizontally arranged ineach of the plurality of dots.
 10. The method of claim 3, wherein eachof the plurality of dots is divided into a plurality of sub-dots andWherein the plurality of class identifiers are assigned to the pluralityof sub-dots.
 11. A method of reconstructing a three dimensional imageusing a structured light pattern system, the method comprising:generating a projected image to be projected onto an object from areference image, wherein the reference image includes a plurality oftiles, wherein each of the tiles includes a plurality of dots each ofwhich is assigned to one of a plurality of class identifiers; generatinga captured image from the projected image onto the object; extracting aclass identifier of an observed pixel on the captured image; andcalculating a first relative position of the observed pixel in a tiledomain of the captured image; calculating a second relative position ofone of the plurality of dots in a tile domain of the reference imageusing the extracted class identifier and a y coordinate of the observedpixel; and calculating a disparity of the observed pixel using the firstrelative position and the second relative position.
 12. The method ofclaim 11, wherein the plurality of class identifiers each is assigned toone of the plurality of dots loft to right and row-by-row within thetile domain of the reference image, wherein the plurality of classidentifiers have continuously-increasing values from a first corner ofthe tile domain of the reference image to a second corner of the tiledomain of the reference image facing diagonally the first corner. 13.The method of claim 11, wherein the calculating of the first relativeposition includes: translating the observed pixel into the tile domainof the captured image to generate the first relative position in thetile domain of the captured image.
 14. The method of claim 11, whereinthe calculating of the second relative position is performed by using afunction of which an input includes the extracted class identifier andthe y-coordinate of the observed pixel and an output is the secondrelative position, and wherein the function represents a distribution ofthe plurality of class identifiers on the tile domain of the referenceimage.
 15. The method of claim 14, wherein the plurality of dots of thereference image is rotated at a predefined rotation angle in theprojected image, and wherein the input of the function further includesthe predefined rotation angle.
 16. A method of reconstructing a threedimensional image using a structured light pattern system, the methodcomprising: generating a reference image including a plurality of tiles,wherein each of the tiles includes a plurality of dots each of which isassigned to one of a plurality of class identifiers; projecting aprojected image generated from the reference image onto an object havinga three-dimensional surface; capturing the projected image onto theobject to generate a captured image; extracting a class identifier froman observed pixel on the captured image; and translating the observedpixel into a tile domain of the captured image to generate a firstrelative position in the tile domain of the captured image; calculatinga second relative position of one of the plurality of dots in a tiledomain of the reference image using the extracted class identifier;calculating a disparity of the observed pixel using the first relativeposition and the second relative position; and generating athree-dimensional profile of the object using the disparity.
 17. Themethod of claim 16, wherein the calculating of the second relativeposition of the reference image is performed using a functionrepresenting a distribution of the plurality of class identifiers on thetile domain of the reference image, wherein the extracted classidentifier is a first input of the function, and wherein a row number ofthe observed pixel in the captured image is a second input of thefunction.
 18. The method of claim 17, wherein the plurality of dots ofthe reference image is rotated at a predefined rotation angle in theprojected image, and wherein the predefined rotation angle is a thirdinput of the function.
 19. The method of claim 17, wherein a dotsampling rate is a fourth input of the function, and wherein the dotsampling rate corresponds to a number of pixels horizontally arranged ineach of the plurality of dots.
 20. The method of claim 16, wherein thegenerating of the reference image includes: assigning each of theplurality of class identifiers to one of the plurality of dots, whereininteger values of the plurality of class identifiers increasesuccessively.